Context-based Information Fusion: A survey and discussion

This survey aims to provide a comprehensive status of recent and current research on context-based Information Fusion (IF) systems, tracing back the roots of the original thinking behind the development of the concept of \u201ccontext\u201d. It shows how its fortune in the distributed computing world eventually permeated in the world of IF, discussing the current strategies and techniques, and hinting possible future trends. IF processes can represent context at different levels (structural and physical constraints of the scenario, a priori known operational rules between entities and environment, dynamic relationships modelled to interpret the system output, etc.). In addition to the survey, several novel context exploitation dynamics and architectural aspects peculiar to the fusion domain are presented and discussed

An Expanded Framework for Situation Control

There is an extensive body of literature on the topic of estimating situational states, in applications ranging from cyber-defense to military operations to traffic situations and autonomous cars. In the military/defense/intelligence literature, situation assessment seems to be the sine qua non for any research on surveillance and reconnaissance, command and control, and intelligence analysis. Virtually all of this work focuses on assessing the situation-at-the-moment; many if not most of the estimation techniques are based on Data and Information Fusion (DIF) approaches, with some recent schemes employing Artificial Intelligence (AI) and Machine Learning (ML) methods. But estimating and recognizing situational conditions is most often couched in a decision-making, action-taking context, implying that actions may be needed so that certain goal situations will be reached as a result of such actions, or at least that progress toward such goal states will be made. This context thus frames the estimation of situational states in the larger context of a control-loop, with a need to understand the temporal evolution of situational states, not just a snapshot at a given time. Estimating situational dynamics requires the important functions of situation recognition, situation prediction, and situation understanding that are also central to such an integrated estimation + action-taking architecture. The varied processes for all of these combined capabilities lie in a closed-loop “situation control” framework, where the core operations of a stochastic control process involve situation recognition—learning—prediction—situation “error” assessment—and action taking to move the situation to a goal state. We propose several additional functionalities for this closed-loop control process in relation to some prior work on this topic, to include remarks on the integration of control-theoretic principles. Expanded remarks are also made on the state of the art of the schemas and computational technologies for situation recognition, prediction and understanding, as well as the roles for human intelligence in this larger framework

MAVERICK: A Synthetic Murder Mystery Network Dataset to Support Sensemaking Research

AbstractThe MAVERICK dataset was created to support a series of empirical studies looking at the effectiveness of network visualizations intended to support information foraging and human sensemaking within the domain of counterinsurgency intelligence analysis. This synthetic dataset is structured as a forensic mystery with the central goal of solving a fictional murder. The dataset includes 181 text-based reports, with additional media included with some messages as attachments, collected from various sources of varying reliability. The reports are framed as being collected from the perspective of a reporter investigating the murder through interviews with suspects and observations taken at the site the murder. The dataset includes intentional and unintentional deception along with calculated source reliabilities based on available evidence. The dataset is dynamic in nature, as the information in the dataset evolves and expands over a simulated period of time. This is done to both to simulate a real-world scenario, and to allow for evolutionary tasks and experiments to be performed using the dataset. The dataset is designed to be complex enough to simulate a real-world, while remaining accessible to individuals without experience in a specific domain of interest. This meant that it had to be on a topic that did not require prior domain knowledge to understand available information or to understand what strategies should be applied during analysis of the dataset. The solution to these challenges was the development of a fictional murder mystery. The plot involves a murder that took place over the course of a weekend with several possible suspects at a large private estate. This scenario allowed for a great deal of complexity; however, it was also a subject matter that could be easily understood by participants without prerequisite domain experience

Research Opportunities in Contextualized Fusion Systems. The Harbor Surveillance Case

Proceedings of: International Workshop of Intelligent Systems for Context-Based Information Fusion (ISCIF 2011) associated to 11th International Work-Conference on Artificial Neural Networks, IWANN, Torremolinos-Málaga, Spain, June 8-10, 2011.The design of modern Information Fusion (IF) systems involves a complex process to achieve the requirements in the selected applications, especially in domains with a high degree of customization. In general, an advanced fusion system is required to show robust, context-sensitive behavior and efficient performance in real time. It is necessary to exploit all potentially relevant sensor and contextual information in the most appropriate way. Among modern applications for IF technology is the case of surveillance of complex harbor environments that are comprised of large numbers of surface vessels, high-value and dangerous facilities, and many people. The particular conditions and open needs in the harbor scenario are reviewed in this paper, highlighting research opportunities to explore in the development of fusion systems in this area.This work was supported in part by Projects CICYT TIN2008-06742-C02-02/TSI, CICYT TEC2008-06732-C02-02/TEC and CAM CONTEXTS S2009/TIC-1485.Publicad

Overview of contextual tracking approaches in information fusion

Proceedings of: Geospatial InfoFusion III. 2-3 May 2013 Baltimore, Maryland, United States.Many information fusion solutions work well in the intended scenarios; but the applications, supporting data, and capabilities change over varying contexts. One example is weather data for electro-optical target trackers of which standards have evolved over decades. The operating conditions of: technology changes, sensor/target variations, and the contextual environment can inhibit performance if not included in the initial systems design. In this paper, we seek to define and categorize different types of contextual information. We describe five contextual information categories that support target tracking: (1) domain knowledge from a user to aid the information fusion process through selection, cueing, and analysis, (2) environment-to-hardware processing for sensor management, (3) known distribution of entities for situation/threat assessment, (4) historical traffic behavior for situation awareness patterns of life (POL), and (5) road information for target tracking and identification. Appropriate characterization and representation of contextual information is needed for future high-level information fusion systems design to take advantage of the large data content available for a priori knowledge target tracking algorithm construction, implementation, and application.Publicad

High-Level Information Fusion in Visual Sensor Networks

Information fusion techniques combine data from multiple sensors, along with additional information and knowledge, to obtain better estimates of the observed scenario than could be achieved by the use of single sensors or information sources alone. According to the JDL fusion process model, high-level information fusion is concerned with the computation of a scene representation in terms of abstract entities such as activities and threats, as well as estimating the relationships among these entities. Recent experiences confirm that context knowledge plays a key role in the new-generation high-level fusion systems, especially in those involving complex scenarios that cause the failure of classical statistical techniques –as it happens in visual sensor networks. In this chapter, we study the architectural and functional issues of applying context information to improve high-level fusion procedures, with a particular focus on visual data applications. The use of formal knowledge representations (e.g. ontologies) is a promising advance in this direction, but there are still some unresolved questions that must be more extensively researched.The UC3M Team gratefully acknowledges that this research activity is supported in part by Projects CICYT TIN2008-06742-C02-02/TSI, CICYT TEC2008-06732-C02-02/TEC, CAM CONTEXTS (S2009/TIC-1485) and DPS2008-07029-C02-02

Fusion Based Safety Application for Pedestrian Detection with Danger Estimation

Proceedings of: 14th International Conference on Information Fusion (FUSION 2011). Chicago, Illinois, USA 5-8 July 2011.Road safety applications require the most reliable data. In recent years data fusion is becoming one of the main technologies for Advance Driver Assistant Systems (ADAS) to overcome the limitations of isolated use of the available sensors and to fulfil demanding safety requirements. In this paper a real application of data fusion for road safety for pedestrian detection is presented. Two sets of automobile-emplaced sensors are used to detect pedestrians in urban environments, a laser scanner and a stereovision system. Both systems are mounted in the automobile research platform IVVI 2.0 to test the algorithms in real situations. The different safety issues necessary to develop this fusion application are described. Context information such as velocity and GPS information is also used to provide danger estimation for the detected pedestrians.This work was supported by the Spanish Government through the Cicyt projects FEDORA (GRANT TRA2010- 20225-C03-01 ) , VIDAS-Driver (GRANT TRA2010-21371-C03-02 ).Publicad

Strategies and Techniques for Use and Exploitation of Contextual Information in High-Level Fusion Architectures

Proceedings of: 13th Conference on Information Fusion (FUSION 2010): Edinburgh, UK. 26-29 July 2010.Contextual Information is proving to be not only an additional exploitable information source for improving entity and situational estimates in certain Information Fusion systems, but can also be the entire focus of estimation for such systems as those directed to Ambient Intelligence (AI) and Context-Aware(CA) applications. This paper will discuss the role(s) of Contextual Information (CI) in a wide variety of IF applications to include AI, CA, Defense, and Cyber-security among possible others, the issues involved in designing strategies and techniques for CI use and exploitation, provide some exemplars of evolving CI use/exploitation designs on our current projects, and describe some general frameworks that are evolving in various application domains where CI is proving critical.The UC3M Team gratefully acknowledge that this research activity is supported in part by Projects CICYT TIN2008-06742-C02-02/TSI, CICYT TEC2008-06732- C02-02/TEC, CAM CONTEXTS (S2009/TIC-1485) and DPS2008-07029-C02-02. UC3M also thanks Prof. James Llinas for his helpful comments during his stay, which has been supported by the collaboration agreement ‘Chairs of Excellence’ between University Carlos III and Banco Santander. The US/UB Team gratefully acknowledge that this research activity is supported by a Multidisciplinary University Research Initiative (MURI) grant (Number W911NF-09-1-0392) for “Unified Research on Networkbased Hard/Soft Information Fusion”, issued by the US Army Research Office (ARO) under the program management of Dr. John LaveryPublicad

Implementing Dempster-Shafer Theory for property similarity in Conceptual Spaces modeling

Previous work has shown that the Complex Conceptual Spaces − Single Observation Mathematical framework is a useful tool for event characterization. This mathematical framework is developed on the basis of Conceptual Spaces and uses integer linear programming to find the needed similarity values. The work of this paper is focused primarily on space event characterization. In particular, the focus is on the ranking of threats for malicious space events such as a kinetic kill. To make the Conceptual Spaces framework work, the similarity values between the contents of observations on the one hand and the properties of the entities observed on the other needs to be found. This paper shows how to exploit Dempster-Shafer theory to implement a statistical approach for finding these similarities values. This approach will allow a user to identify the uncertainty involved in similarity value data, which can later be propagated through the developed mathematical model in order for the user to know the overall uncertainty in the observation-to-concept mappings needed for space event characterization

Blinded Predictions and Post Hoc Analysis of the Second Solubility Challenge Data: Exploring Training Data and Feature Set Selection for Machine and Deep Learning Models

Accurate methods to predict solubility from molecular structure are highly sought after in the chemical sciences. To assess the state of the art, the American Chemical Society organized a "Second Solubility Challenge"in 2019, in which competitors were invited to submit blinded predictions of the solubilities of 132 drug-like molecules. In the first part of this article, we describe the development of two models that were submitted to the Blind Challenge in 2019 but which have not previously been reported. These models were based on computationally inexpensive molecular descriptors and traditional machine learning algorithms and were trained on a relatively small data set of 300 molecules. In the second part of the article, to test the hypothesis that predictions would improve with more advanced algorithms and higher volumes of training data, we compare these original predictions with those made after the deadline using deep learning models trained on larger solubility data sets consisting of 2999 and 5697 molecules. The results show that there are several algorithms that are able to obtain near state-of-the-art performance on the solubility challenge data sets, with the best model, a graph convolutional neural network, resulting in an RMSE of 0.86 log units. Critical analysis of the models reveals systematic differences between the performance of models using certain feature sets and training data sets. The results suggest that careful selection of high quality training data from relevant regions of chemical space is critical for prediction accuracy but that other methodological issues remain problematic for machine learning solubility models, such as the difficulty in modeling complex chemical spaces from sparse training data sets